A Neuro - Dynamic Programming Approach

We formulate the call admission control problem for a single link in an integrated service environment as a Markov Decision Problem. In principle , an optimal admission control policy can be computed using methods of Dynamic Programming. However, as the number of possible states of the underlying Markov Chain grows exponentially in the number of customer classes, Dynamic Programming algorithms for realistic size problems are computation-ally infeasible. We try to overcome this so-called \curse of dimensionality" by using methods of Neuro-Dynamic Programming (NDP for short). NDP employs simulation-based algorithms and function approximation techniques to nd control policies for large-scale Markov Decision Problems. We apply two methods of NDP to the call admission control problem: the TD(0) algorithm and Approximate Policy Iteration. We assess the performance of these methods by comparing with two heuristic policies: a policy which always accepts a new customer when the required resources are available, and a threshold policy.